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Abstract 


Let  {Xn,n>0}  and  {Yn,n>0}  be  two  stochastic  processes  such  that 

Yn  depends  on  Xn  in  a  stationary  manner,  i.e.  P(YneA|Xn)  does  not  depend 

on  n.  Sufficient  conditions  are  derived  for  Yn  to  have  a  limiting  distribution. 

If  X  is  a  Markov  chain  with  stationary  transition  probabilities  and 
n 

Y  =  f(X  ,...,Xn+k)  then  Yn  depends  on  Xn  is  a  stationary  way.  Two  situations 
are  considered: (i )  {Xn,n>0}  has  a  limiting  distribution  (ii)  (Xn,n>0) 

does  not  have  a  limiting  distribution  and  exits  every  finite  set  with 
probability  1.  Several  examples  are  considered  including  that  of  a  non- 
homogeneous  Poisson  process  with  periodic  rate  function  where  we  obtain  the 
limiting  distribution  of  the  interevent  times. 


Key  Words:  Markov  Chains,  Limiting  Distributions,  Periodic  Nonhomogeneous 
Poisson  Processes. 
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A  N  VvV  > 


/  •• 


Si.  Let  {Xn,n>0}  be  a  discrete  time  Markov  chain  with  stationary  transition 


probabilities.  Consider  a  process  {Yp,n>0}  defined  by 


(1.1) 


Y  =  f (X  ,X  . . X  A. )  . 

n  n  n+1  n+k' 


It  snould  be  emphasized  that  f  and  k  do  not  depend  upon  n.  In  this  paper 
we  address  the  following  question:  under  what  conditions  on  the  Markov 
chain  {Xn,n>0}  and  the  function  f  will  {Yp,n>0}  have  a  limiting  distribution? 

As  a  first  example  of  the  above  situation,  suppose  {Xp,n>0}  is  a  random 

walk,  i.e.  XQ  =  0  and  Xn  =  +  Z2  +  . . .  +  Zp  where  {Zn,nsO}  is  a  sequence 

of  i.i.d.  random  variables.  Let  Yn  =  f(Xn>Xn+^)  *  Xn+^  -  Xp.  Here  {Xn,n>0} 

itself  does  not  possess  a  limiting  distribution  (except  in  the  trivial  case 

where  Z  =  0  w.p.l  for  all  n^O),  but  {Y  ,n;>0}  does  have  a  limiting  distribution 
n  n 

(in  fact  it  is  a  sequence  of  i.i.d.  r.v.) 

As  a  second  example  consider  a  nonhomogeneous  Poisson  process  with  rate 
function  x(t).  Suppose  that  x(t)  is  a  periodic  function  of  t.  Let  X^  be 
the  n-th  event  occurrence  time  and  let  Yn  =  f(Xn,Xn+^)  *  Xp+-|  -  Xn  be  the 
n-th  interevent  time.  Now,  {Xn ,n >0 }  is  a  transient  Markov  chain,  but  due  to 
the  periodic  nature  of  x(t),  one  expects  {Yn,nsO}  to  have  a  limiting  distribution. 
This  example  is  treated  in  detail  in  example  2  of  section  3. 

Though  we  have  stated  the  problem  for  a  Markov  chain  {Xn,ns0}  and  its 
functional  process  {Yn>nsO},  the  theory  that  we  develop  in  the  next  section, 
in  fact,  does  not  use  the  Markov  property  of  {Xp,n>0}  or  the  functional 
dependence  of  Yn  on  xn»‘ • • ’*n+k’  9enera^  structure  that  we  assume  is  as 
follows: 

Let  {X  ,n'0)  be  a  sequence  of  random  variables  and  {Y  ,n>0}  be  another 
n  n 


sequence  of  random  variables  with  the  property  that 

(1.2)  P(YneA|Xn)  C*0eS  not  ^ePen<^  on  n- 

We  derive  a  sufficient  condition  under  which  {Yn,ns0}  has  a  limiting  distribu¬ 
tion.  Notice  that  if  {Xn ,nsO>  is  a  Markov  chain  with  stationary  transition 
probabilities  and  Yn  is  defined  by  eq.(l.l)  then  condition  (1.2)  is  automatically 
satisfied. 

In  theorem  1  in  the  next  section  we  state  a  sufficient  condition  for  the 
existence  of  limiting  distribution  of  {Yn,n20}  .  We  also  show  with  an  example 
that  the  condition  is  not  necessary.  In  the  general  setting  of  the  theorem  it 
seems  difficult  to  derive  a  useful  necessary  condition.  Even  though  the 
condition  stated  in  theorem  1  is  only  sufficient,  it  is  nonetheless  a 
powerful  tool  to  unify  several  cases  as  Is  shown  by  the  material  in  sections  3 
and  4. 

In  section  3  we  consider  stochastic  processes  {Xn,n>0}  having  a  limiting 
distribution.  From  theorem  1  we  obtain  proposition  1  which  gives  the  limiting 
distribution  of  {Yn,n>0}.  In  section  4  we  consider  countable  state  space 
stochastic  processes  {Xn,n>0}  which  do  not  possess  a  limiting  distribution  and 
have  the  property  that  {Xn,nsO}  exits  every  finite  set  with  probability  1.  In 
proposition  2  we  state  a  sufficient  condition  for  {Yn>n>0}  to  have  a  limiting 
distribution  in  this  case.  Several  examples  are  given  to  illustrate  both  the 
propositions.  Although  the  results  are  derived  for  general  stochastic  processes, 
the  examples  deal  with  Markov  chains  {Xn,n>0}.  This  is  purely  for  the  sake  of 
computational  ease. 

Limit  theorems  have  been  studied  in  the  literature  for  the  case  when  lXn,n  01 
is  a  Markov  chain  and  Yn  =  f(Xn).  These  limit  theorems  deal  with  the  partial 


3  °f  22  *  7| 

sums  x Y-.  (See  [3]).  We  are  not  aware  of  any  theorems  for  {Y  ,n>0)  itself. 

i=l  1  n 

Another  specific  problem  that  has  been  addressed  in  the  literature  is  :  If 

{Xn,n>0}  is  a  Harkov  chain,  under  what  conditions  is  Yn  =  f (Xn )  also  a  Markov 

chain?  (See  [4]).  In  this  paper  we  are  interested  in  the  limiting  behaviour 

and  not  the  Markov  nature  of  {Y  ,n>0}. 


Let  (:„,f-,P)  be  a  probability  space  and  let  (E,E)  be  a  measurable 
space.  Let  {X(),  n>0}  be  a  sequence  of  (E,E)  valued  random  variables  on 
(u,F,P).  Let  S  be  a  complete  separable  metric  space  and  let  B(S)  be  its 
Borel  o-field  and  M(S)  be  the  space  of  probability  measures  on  (S,8(S)). 
Let  fYn>  n>0}  be  a  sequence  of  (S,8(S))  valued  random  variables  on  (ft,F,P) 
such  that 

(2-D  P(Yn£A|a(Xn))  =  p^  (A) 

where  p.(-)  is  a  mapping  from  Ex8(S)  1R  such  that 

(2.2)  for  all  xeE,  px(-M(S) 

(2.3)  for  all  AeS(S),  p  (A)  is  a  measurable  function  on  (E,E). 

Here,  o(Xn)  denotes  the  smallest  o-field  on  ft  with  respect  to  which  Xn  is 
measurable. 

Let  S*  =  M(S)  be  equipped  with  the  topology  of  weak  convergence. 

(See  [  1],  [  6]).  Recall  that  un  -*■  u  in  S*  iff  for  all  bounded  continuous 
functions  f  on  S  / fdu  -►  /fdp  .  S*  Itself  is  a  complete  separable 


metric  space  under  this  topology.  (See  [6]).  Let  8(S*)  and  M(S*)  denote 
the  Borel  a-field  on  S*  and  the  space  of  probability  measures  on  (S*,8(S*)) 
respectively.  M(S*)  is  also  equipped  with  the  topology  of  weak  convergence. 

Using  eq.(2.2)  and  (2.3)  it  can  be  shown  that  x  px  is  a  measurable 

mapping  from  (E,E)  into  (S*,8(S*))  and  hence  py  is  a  (S*»B(S* ) )  valued  random 

An 

variable.  Let  r  denote  the  distribution  of  pY  *  i.e.  for  BeS(S*) 
n  *n 

(2.4)  r  (B)  =  P(px  (-)eB)  . 

n 

With  these  notations  we  have  the  following 

Theorem  1:  Suppose 


(2.5)  rn  r  (say)  weakly  in  M(S*), 

then  Yn  converges  in  distribution  to  a  measure  veS*  given  by 

(2.6)  v(A)  *  /  u(A)dr(u)  (AtB(S)) . 

S* 

Proof:  Let  f  be  a  bounded  continuous  function  on  S  and  let  F:S*  -+  1R  be  defined 
by 

(2.7)  F(u)  =  Jfdy  . 


Then,  by  the  definition  of  weak  convergence,  it  follows  that  F  is  a  bounded 
continuous  function.  Thus  from  eq.  (2.5),  we  get 
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(2.8) 


/F(vi)drn(u)  -*■  /F(p)dr(u) 


Now 


(2.9) 


/ F(u)  drn(y)  =  E[F(px  (•))] 


=E[/f(z)Pv  (dz)] 
n 


From  eqs.  (2.1),  (2.2)  and  (2.3)  it  follows  that 


E(f(Yn)'o(Xn))  =  ^f(z)pX  (dz) 


and  hence  by  eq.  (2.9) 


(2.10)  /F(u)drn(y)  =  E[E[f(Yn)|a(Xn)]] 

=  E[f(Yn)] 

Also, 

(2.11)  /F(p)dr (P)  -  /  {/  f(z)dM(z)j  dr(p) 

S*  S 

=  Jf(z)dv(z) 

S 

The  last  equality  in  eq.  (2.11)  follows  from  the  definition  of  v  for  a  simple 
function  f  and  then,  by  the  usual  arguments  for  a  general  function  f. 


Thus  from  eqs.  (2.8),  (2.10)  and  (2.11)  we  have,  for  all  bounded 
continuous  functions  f  on  S, 

(2.12)  E(f(Yn))  -  / f(z)dv(z) 

and  hence,  Yn  converges  in  distribution  to  v.  0 

Remark:  The  condition  (2.5)  is  not  necessary  as  is  shown  by  the  following 
example.  Let  E  =  {0,^,1}  and  let  (Xn,  nsO)  be  such  that,  for  n^O, 

P(X2n  «  0)  =  P(X2n  =  1)  =  is 
p(X2n+l  *  ^  = 

Let  S  =  {0,1}  and  define  Px(‘)  as  follows: 

Px({0})  =  X  =  1  -  Px ( { 1 > )  (xeE)  . 

Let  { Yn ,  n>0}  be  S  valued  random  variables  satisfying  eq.  (2.1).  Then, 
for  all  n>0, 

P(Y„  *  0)  =  EX„  .  % 

and  hence,  trivially,  Yn  converges  in  distribution.  However,  it  is  easy 
to  see  that  rn  does  not  converge  weakly. 


The  above  theorem  provides  a  general  framework  to  study  the  problems 
mentioned  in  the  introduction.  In  later  sections,  we  discuss  several  special 
problems  and  obtain  sufficient  conditions  for  convergence  in  distribution 
of  fYn,  n>0).  In  each  of  these  cases,  the  results  could  be  proved  by 
alternate  techniques,  but  the  above  theorem,  whose  proof  is  simple,  provides 
a  unified  view  of  the  problem. 

ii3.  In  this  section  we  restrict  ourselves  to  the  class  of  stochastic  processes 
{Xn,  n>0)  which  possess  limiting  distributions,  i.e.  Xn  converges  in  distribution 
to  some  measure  n  as  n-*=°.  In  the  framework  of  section  2,  suppose  that  E  is 
a  metric  space  and  E  is  its  Borel  a-field.  The  following  is  an  easy  consequence 
of  theorem  1 : 

Proposition  2:  Suppose  that 

(3.1)  x  -►  p^  is  a  continuous  function  from  E  into  S*  and 

(3.2)  Xn  converges  in  distribution  to  some  measure  it  on  (E,E). 

Then  converges  in  distribution  to  a  measure  v  on  (S,8(S))  defined  by 

(3.3)  v(A)  =  /px(  A)dir(x)  (AeB(S))  . 

E  x 

Proof:  Eqs.  (3.1)  and  (3.2)  imply  that  p^  converges  in  distribution  to 


8  of  22 


r  =  uo(p  )  ,  which  is  the  same  as  +  r  weakly.  Hence  Yn  converges  in 

distribution  to  v  given  by,  for  Ae8(S), 


v(A)  =  /  y(A)dr(y) 

S* 

=  /  p(A)diro(p  )  1  (p ) 
S* 


=  /pJA)diT(x). 


We  now  give  two  examples  illustrating  the  above  result.  In  the  remaining 
paper,  all  finite  or  countable  sets  will  be  equipped  with  the  discrete 
topology.  Our  first  example  is  that  of  a  positive  recurrent  Markov  chain. 


Example  1.  Let  E  =  {0,1,2,...}  and  let  (Xn>  n>0}  be  an  E-valued  Markov  chain 
with  stationary  transition  probabilities  given  by 


P(X,  =  k+l|Xn  =  k)  =  b  (k>.l)  , 


P(X1  =  k-1 jXQ  =  k)  =  a  =  1  -  s  (k>l )  , 


P(X,  =  1  I  Xn  =  0)  =  1. 


Let  S  =  {-1,1}  and  Yn  =  xn+-|-xn-  Then  eq.  (2.1)  is  satisfied  with 


V  V .V.V.V?.- V.V.V  JV.V/ V V. \* VAr-V-V  .•_*  .•  •.•  valVv: 


Pk(0})  =  8 


(k>l)  , 

Pk({-H)  =  a  (k>l)  , 

p0(n»  »  i  . 

Since  the  sets  E  and  S  are  equipped  with  discrete  topology,  conditions  (2.3) 
and  (3.1)  are  trivially  satisfied.  Now  suppose  a>B>0.  Then  {Xn,n>0}  is 
positive  recurrent  and  its  stationary  distribution  is  given  by 


\  ■  ES-  <f)k  (k2l)  • 

Hence  {XR ,  n>0>  converges  in  distribution  to  n.  Hence,  by  proposition  2, 
Yn  converges  in  distribution  to  v  where 

oo 

v({l })  =  Lim  P(Yn  *  1)  -  e  Pk({l})*k  = 
n->°°  k=0 


and 


v( f 0 } )  =  Lim  P(Yn  =  -1)  =  b  . 

n-*-™ 

Thus,  Yp  converges  in  distribution  to  Note  that  Yn,  the  increment  of 

the  (X  ,n-’0>  process,  has  a  stationary  distribution  independent  of  a! 
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Our  next  example  is  that  of  a  non-homogeneous  Poisson  process. 

Example  2.  Let  {N(t),  t>0}  be  a  non-homogeneous  Poisson  process  with 
strictly  positive  rate  function  x(t).  Assume  that  x(t)  is  periodic  with 
period  t,i .e. , 

x(t  +  t)  =  x ( t )  for  all  t>0. 

Let  ZQ  -  0  and  be  the  time  that  the  n-th  event  occurs,  i.e. 

zn+i  =  ^f  (UZn  :  N{t)  >  N(Zn)}  n>0. 

and  let  Yn  =  Zn+1-Zn  be  the  nth  interevent  time.  Now,  for  0<v1  <  v?  •_... 

^  vn  <  u, 

P(zn+1  >ulz,  -  v,,...zn  .  v„) 

=  P(N(u)  =  n  |  N(s):0^s«  vn,N(vn)  =  n) 

=  exp  (-/u  x(s)ds)  . 
vn 

Hence  {Zn,n>0}  is  a  Markov  chain.  As  Zp  increases  with  n,  it  is  a  transient 
Markov  chain. 

Let  [x]  denote  the  largest  integer sx  and  define 
xn  ■  zn  -  • 


Also,  for  t>0  define 
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A(t)  -  /Jx(s)ds. 

Now  for  0  <  y  <  «  , 

p(Yn  >  yiZn)  =  R(Zn+l  >  Zn  + 

=  exp  (-(A(Zn  +  y)  -  A(Zn) ) ) 

=  exp  ( - ( A( X  +  y)  -  A(X  ))) 
n  11 

For  0  *  x  <  t,  let  px  denote  the  probability  measure  on  (1R  +  ,B(1R+))  given  by 
(3.4)  Px((y,°°))  =  exp  (-(a(x  +  y)  -  a(x) ) )  . 

Then  for  Ac8(lR+),  we  have 


I 


l 


■i 


P(YncA|Zn)  =  P(YneA|Xn)  =  P)(  (A). 

n 

Let  E  =  [0,t) .  Equip  E  with  a  topology  which  makes  the  mapping 
x  >  exp  ( i 2nx/ t )  a  homeomorphism  from  E  onto  the  unit  circle  in  the  complex 
plane.  Thus 


i _ 


xn  -»  x  iff  exp(i2irxn/T)  exp(i2Trx/t)  . 


Under  this  topology  E  is  a  complete  separable  metric  space  and  it  can  be  easily 
checked  that  the  mapping  x  -*■  px  in  eq.  (3.4)  from  E  -*■  M(1R)  is  continuous. 
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We  shall  now  show  that  {Xn,  n  >  0}  converges  in  distribution  to  some 
measure  on  E.  In  fact,  it  can  be  shown  that  {Xn>  n  >  0}  is  a  Markov  chain  on 
E  with  stationary  transition  probabilities  given  by 

p(Xn+i>y|xn  =  x)  =  eA(x)  (e'A(y)-B)/(l-B)  if  0  <  x  <  y  <  T 

(3.5)  =  1  -  BeA^  (1  -  e-A*y))/(l  -  B) 

if  0  <y  <  x  m  . 


where  B  =  exp(-A(r))  . 

Let  f(*,x)  be  the  conditional  density  of  Xn+1  given  X^  *  x.  Then  from  eq. 
(3.5)  we  get 


(3.6)  f(y,x) 


x(y)  exp(A(x)  -  A(y ) )/( 1 -B)  if  0  <  x  <  y  <t 

B>(y)  exp(A(x)  -  A(y))/(1-B)  if  0  s  y  <  x  <  i . 


By  a  slight  modification  of  the  arguments  in  Example  (b)  of  VII. 7  of  Feller 
Vol  2  [  2],  one  can  show  that  { Xn ,  n  ^  0)  has  a  limiting  distribution.  Let 
g(-)  be  the  limiting  density  of  Xn-  Then  g(-)  satisfies 

g(y)  =  /^(y»x)g(x)dx. 


Thus 

g(y)  -  MW  «»P(-MW)  [/y  eA<“)g(x)dx  +  B  j'  eA<x)g(*)dx] 


Now  let  h(y)  =  g(y)  exp(A(y)).  Then 


h(y)  =  ^  [J*h(x)dx  +  B fry  h(x)dx]  . 

Differentiating  the  above  equation  we  get 

h'(y)  =  (i|y^+  *(y))  My) 

which  has  a  unique  solution 

h(y)  =  x(y)  exp(A(y))  . 

Hence  we  get  g(y)  =  Cx(y),  the  constant  of  integration  C  is  found  by  using 


/o  g(y)dy  *  i 


Thus  g(y)  =  x(y)/A(i;)  (0  <  y  <  t)  is  the  limiting  density  of  Xp  . 

Now,  using  proposition  2,  we  get  the  following:  {Yn,  n  >  0}  has  a 
limiting  distribution  given  by 

(3.7)  L,m  P(Vn  ’ 

=  /J  exp(-(A(x  +  y)  -  a(x) ) )  x(x)  dx/A(t)  . 


A  .  •  ••  A 


t  '  ■ 


•  .''A, 
•V'A 
A*.*» 


i4.  In  this  section  we  consider  the  case  in  which  {X  ,  n>0)  does  not 

n 

converge  in  distribution,  but  {Yn>n>0}  does.  First  we  give  a  general  result 
and  then  consider  some  specific  examples. 

Suppose  that  E  is  a  countable  set  and  that 

(4.1)  Lim  P(XneB)  =  0 

n-H» 

for  all  finite  subsets  B  of  E.  Furthermore,  suppose  that  there  exists  a 
partition  { E j »  jeJ}  of  E,  where  either  J  =  {l,2,...,k}  or  J  =  {1,2,3,... I, 
and  each  E-  is  a  countable  set  such  that 

(4.2)  Lim  P(X  eE.)  =  a.  exists 

n-**>  n  J  J 

and 

(4.3)  Z  a.  =  1 

jeJ  J 

Now,  let  {x^}  i  >  0  be  an  enumeration  of  E.  and  suppose  that 

(4.4)  Lim  p  =  v- 

i-«°  i  j  J 

exists.  (Recall  that  M(S)  is  equipped  with  the  topology  of  weak  convergence.) 
With  this  structure  we  get  the  following 


••  s  %  ^  • 
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r  M*  »*  I 

.1 
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Proposition  3-  Let  eqs.  (4.1)  -  (4.4)  hold.  Then  {Yn,na:0}  converges  in 
distribution  to  v  given  by 


(4.5) 


v(A)  =  Z  Uj (A)a. 
jeJ  J  3 


(AeB(S) ) . 


Proof:  The  desired  conclusion  will  follow  from  Theorem  1  if  we  show  that 


(4.6) 


rn  ->  r  in  M(S*) 


where  r  is  given  by 


r(B)  =  z  lJuJa. 

jcO  B  J  0 


(BeB(S*)). 


Let  f  be  a  bounded  continuous  function  on  S*.  Then 


(4.7) 


j/fdr  -  /fdr|  -  |E(f(px  ))  -  z  f(pi)ai ! 

n  jeJ  J  3 


<  Z  |Ef(p  )  1  -  f(v)a| 

jeO  An  {V*\j}  J  3 
"  *  |E(f (px  )  -  f(ui))  lfX  E  J 

JeJ  *n  3  {Vhj} 


+  £  |f(pi)||P(Xn£E.)  -  a. 

j€J  3  n  J  J 


£  i  v"  (say)  . 

jeJ  3  jeJ  J 


■*.  •*  .•  .* ; .*  v  ,v  -.*  •/ 
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Let  a"  =  P(X  €E.).  Then 

J  >1  J 


(4.9)  Z  |a-  -  a. 

jcJ  J  3 


Z  a.  +  Z  a-  -  2  Z  Aa- 
jeJ  J  jeJ  J  jtj  J  J 


2-2  Z  a.Aa. 
j  £  J  J  J 


-2-2  I  a.  =  0. 

j«J  3 


(By  dominated  convergence  theorem.)  Since  f  is  bounded  eq.(4.9)  implies  that 


(4.10) 


Lim  Z  V,  =  0 
n-»»  jeJ  J 


Eq.  (4.9)  also  implies  that,  given  e  >  0,  there  exists  a  finite  subset  J 

0  o 

of  J  such  that. 


(4.11) 


Sup  Z  a  .  <  € 

n  jeJ.  3  0 


where  J-j  =  J\JQ  .  Eq.  (4.11)  is  actually  the  assertion  that 
implies  uniform  integrability. 

Next  we  show  that,  for  jeJ 


-  convergence 


(4.12) 


Lim  U.  =  0. 

n-*°  J 


Recall  that  p  -  u.  as  i-*»  and  f  is  continuous  on  S*,  so  that  f(p  )>f(g.) 

J  X . .  1 


V-  .V.i 
•-  >'  < 

*-v 

* .  •  v  < 

-\>V\ 

*  .  *•  v 

*  A  . 


v  • 

•.V.'V 


V. 

1  •  *  ■, 


Thus,  given  £>0,  there  is  an  iQ  such  that,  for  all  i>i0» 


(4-13) 


|f(p,  )  -  f(uJ|<c/2. 

xij  J 


Let  B  =  { x • j  :  l<i<iQ}.  Then  in  view  of  (4.1),  there  is  an  nQ  such 
that  for  n>nQ, 


(4.14) 


P{XneB}  <  e/4K 


where  K  is  such  that  | f | <K.  Then  for  nsnQ,  we  have 


Uj  =  Ef|f(pXn)  -  f(wJ-)11(Xn£E.}} 


E{ if(pXn)  '  f(wj)|{1{XneB)  +  1{XneEj\B}}} 


2KP(Xn£B)  +  f  P(XneEJ \B) 


<  e/2  +  c/2 


(by  eqs.  (4.13)  and  (4.14)) 


This  proves  eq.  (4.12).  Now,  for  a  given  e>0,  define  eQ  =  e/2K.  For  this 
0  get  J0  and  J]  as  above,  so  that  eq.  (4.11)  holds.  Then 
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Since  J.  is  finite,  we  have,  using  eq.  (4.12)  and  (4.15) 


(4.16) 


Lim  sup  z  U. 

n->®  jej  J 


Since  €>0  is  arbitrary,  this  shows  that 


Lim  z  U.  =  0. 
n->«  jeJ  J 

In  view  of  eq.  (4.8),  (4.9)  and (4. 10)  we  get 


Lim  |/fdr_  -  / fdr |  »  0 

n-*°° 


which  implies  Vr  weakly,  and  this  completes  the  proof. 


We  now  take  two  examples  illustratinq  the  above  proposition. 


Example  3.  Consider  the  same  Markov  chain  (X^,  naO)  from  example  1,  and 
suppose  o<e.  Then  (Xn,  n>0}  is  a  transient  Markov  chain  and  hence  eq. 

(4.1)  holds. 

Since  Lim  pk  =  where  m^UJ)  =B  and  ^((-ll)  =  a,  the  condition; 
of  proposition  3  are  satisfied  if  we  take  E]  =  E  and  X.]  =  i  (i>0).  Then 
Yn  =  Xn+T  Xn  conver9es  in  distribution  to  u^.i.e. 


Lim  P(Y  =  1)  =  a,  Lim  P(Yn  =  -1)  =  p. 


*  •  »  ■  .  •  •  •  '  »  ■  .  *  t  •  .  •  *  «  *  ~  m  •  •  •  *  “  »  *  *  •  »  *•  *  »  *  •  *  •  *  •  •  »  •  •  *  •  •  •  •  l  •  •  •  «  *  .*  •  •  *  •  *,*  *,•  *  «  V  * 
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Example  4.  Let  E  =  {(j,i)  :  j,i  integers,  0  <  j  <  i  <»}.  Let  (Xn>  be  a 
Markov  chain  with  state  space  E  and  stationary  transition  probabilities 
given  by 


P{Xn+l  =  (1*  i+1)l  Xn  =  (0‘i)}  =  1  1*0- 


P{Xn+l  =  W+1»  1+1 )  I  xn  =  (□»■»)>  *3  0<j <i <« 


p(xn+1  *  (j-1»  i ) |  xn  =  (j ,i ) }  =  a  0<j <i «*>  . 


Where  a+p=l,  0<B<a<l.  For  this  chain  all  states  are  inessential,  as 
(j»i)  ►  (j+l>  i+l)  but  (j+1,  i+1)  A  (j,i).  Hence  (Xn>  na  0}  is  transient 
and  eq.  (4.1)  is  satisfied. 

Let  {Yn,  n>0}  be  a  real  valued  process  such  that  the  conditional 

distribution  of  Y  given  X  =  (j,i)  is  normal  with  mean  j  +  1  and 

i+l 

variance  1  +2  .  Thus 


j+1 


(j  »i ) 


=  N(j  + 


1 

i+l 


1 

j+1 


2_1) 


2  2 

where  N(p,n  )  denotes  the  normal  distribution  with  mean  n  and  variance  a 

on  (1R ,  B(1R)). 

Now  let  J  =  {0,1 ,2,3,. . . }  and 


=  l(j»i)  :  i^j)  . 
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Also  let  x..  =  (j.j+i)  be  an  enumeration  of  E.  .  Then  from  the  properties  of 

I J  J 

the  normal  distribution  it  follows  that 


Lim  p  =  N(J»  7TT  )  • 
i-**>  *ij  J 


Now  let  Zn  =  j  if  xneEj.  It  is  easily  seen  that  Zn  itself  is  a  Markov 
chain  with  stationary  transition  probabilities  as  described  in  example  1 


Lim  P(X  tE, )  =  Lim  P(1  =  j)  =  v 
n->«>  11  J  n-*«  11  J 


where 


u  =  “lil  TT  -  /l\j  fi>u 

O  2a  ’  j  2o8.  V  U"U’ 


Thus  t  a  1  and  hence  eq.  (4.2)  and  (4.3)  hold.  Hence,  by  proposition  3, 
j=0  J 

{Yn,n^0}  converges  in  distribution  to  a  measure  v  given  by 


V  =  N(0.1)  *  £  Si!  (f)J  NU.jir)  • 

0  * 


f  *»’  •/  %! 

r. 

-V. 
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S5.  The  results  in  sections  2,3  and  4  can  be  extended  to  continuous  time 
stochastic  processes  in  a  straight  forward  manner.  Extension  of  the  results 
in  section  4  to  more  general  state  spaces  seems  possible  but  presents  many 
technical  difficulties.  The  results  in  section  4  also  suggest  a  relation¬ 
ship  between  our  approach  and  the  boundary  theory  for  Markov  chains,  but 
at  this  stage  we  have  not  been  able  to  make  it  precise.  It  should  be 
mentioned  that  the  result  in  section  3  about  the  periodic  non-homogeneous 
Poisson  processes  is  of  interest  in  itself. 

The  problem  considered  in  this  paper  is  of  interest  in  the  theory  of 

partially  observable  processes.  In  this  context  one  can  think  of  (X  ,  n^O) 

n 

as  the  core  process  and  {Y^.n^Ol  as  the  observation  process.  (See  [4]). 

For  example  Xn  may  represent  the  "state"  of  the  internal  components  of  a 
machine  at  tine  n  while  Yp  may  represent  its  "performance",  which  may  be 
the  only  observable  quantity  about  the  machine.  The  limiting  behaviour 
of  the  observation  process  (Yn,  n^o)  is  obviously  of  importance  in  the  design 


of  the  machine. 


